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Abstract 

In the area of inductive learning, generalization is a main operation, and the usual 
definition of induction is based on logical implication. Recently there has been a rising 
interest in clausal representation of knowledge in machine learning. Almost all inductive 
learning systems that perform generalization of clauses use the relation (^-subsumption 
instead of implication. The main reason is that there is a well-known and simple technique 
to compute least general generalizations under (^-subsumption, but not under implication. 
However generalization under (^-subsumption is inappropriate for learning recursive clauses, 
which is a crucial problem since recursion is the basic program structure of logic programs. 

We note that implication between clauses is undecidable, and we therefore introduce a 
stronger form of implication, called T-implication, which is decidable between clauses. We 
show that for every finite set of clauses there exists a least general generalization under 
T-implication. We describe a technique to reduce generalizations under implication of a 
clause to generalizations under (^-subsumption of what we call an expansion of the original 
clause. Moreover we show that for every non-tautological clause there exists a T-complete 
expansion, which means that every generalization under T-implication of the clause is 
reduced to a generalization under (^-subsumption of the expansion. 

1. Introduction 

The topic of this paper is generalization of clauses, which is a central problem in the 
area of Inductive Logic Programming (ILP) (Muggleton, 1991, 1993). ILP can be seen 
as the intersection of inductive machine learning and computational logic. In inductive 
machine learning the goal is to develop techniques for inducing hypotheses from examples 
(ol)servations). By using the rich representation formalism of computational logic (clauses) 
for hypotheses and examples, ILP can overcome the limitations of classical machine learning 
representations, such as decision trees (Quinlan, 1986). 

By using a clausal representation we have the ability to learn all types of hypotheses 
describable in first-order logic, in particular the important class of recursive hypotheses. 
Another advantage of using a clausal representation is that clausal theories are easy to 
manipulate for machine learning algorithms. This is due to that changes to a clausal theory 
by adding or deleting clauses or literals have clear and simple effects on the generality of 
the theory. The reader is referred to two introductions to ILP. one presented by Muggleton 
and De Raedt (1994), and one by Lavrac and Dzeroski (1994). Lavrac and De Raedt (1995) 
present a recent survey of ILP research. 

We use the following definition of induction. A theory (background knowledge) T, a set 
of positive examples {E^ , .... -E^} and a set of negative examples {E^ , • • ■ • E^^; of a target 
concept are given. Then a hypothesis H for the target concept is an inductive conclusion if 
and only if: 
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1) T^E+A...AE+, 

2) TAH\=E+A...AE+,and 

3) T AH ^ E^ y ...y E-. 



In other words, the positive examples should not be a logical consequence of the theory 
alone, but a logical consequence of the theory together with the hypothesis, and no negative 
example should be a logical consequence of the theory and the hypothesis. Using clausal 
representation T, i?, {E^ , . . . , E^} and {E^ , ■ ■ ■ , E^} are sets of clauses. 

In this paper we concentrate on the subproblem in inductive learning of finding a clause 
that is a generalization of a set of positive examples. In other words, finding a clause C 
such that 



We are particularly interested in least general generalizations, since every generalization 
of a set of clauses is also a generalization of the least general generalization of this set of 
clauses. Therefore a least general generalization in some sense represents all generalizations. 
A least general generalization is also consistent with the negative examples whenever there 
exists a consistent generalization. 

The most natural and straightforward basis for generalization is implication, since in- 
duction is defined in terms of logical consequence. Plotkin has described (1970, 1971a) a 
technique for the computation of least general generalizations of clauses under a relation 
called 6-subsumption. This relation has been accorded much interest, and it is often used 
instead of implication, since it is easier to compute. However, there is a difference between 
0-subsumption and implication, which sometimes causes the generalizations obtained by 
Plotkin's technique to be over-generalizations with respect to implication. 

Consider the following clauses in which s denotes the successor function: 



The clause Di is a least general generalization under ^-subsumption (LGG^) of Ci and C2, 
and the clause D2 is a least general generalization under implication (LGGI) of Ci and C2. 
It is clear that Di is strictly more general than D2, both under ^^-subsumption and under 
implication. It is also clear that D2 is more appropriate in a definition of natural number. 

To learn recursive clauses, generalization under 0-subsumption is not very adequate, as 
illustrated above. The ability to learn recursive clauses is crucial, since recursion is the 
basic program structure of logic programs. 

In section 2, we describe the most important results concerning generalization under 
0-subsumption, and present a theoretically study of generalization imder implication. In 
section 3, we present a technique to reduce implication to ^-subsumption based on or- 
introduction of literals. Finally, our results, computational complexity and future work are 
discussed in section 4. 

We assume the reader to be familiar with the basic notions and notations in Logic Pro- 
gramming (Lloyd, 1987) and/or Automatic Theorem Proving (Chang & Lee, 1973; Gallier, 
1986). 



C^E+A...AE+. 
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2. Generalization of Clauses 

In the area of Inductive Logic Programming (ILP), the framework for generalization of 
clauses developed by Plotkin (1970. 1971b. 1971a). has born accorded much interest. In 
this section wo will describe this framework, which is based on a relation known as 6- 
subsumption, and the most important results connected with it. 

Since generalization under 0-subsumption is not sufficient for generalization of recursive 
clauses, as shown in the introduction, wo will study the theory of generalization under 
implication. We note that implicaton between clauses is imdocidable, and we will therefore 
introduce a restricted form of implication, called T-implication. 



2.1 Generalization under 0-subsumption 

Definition A clause C 0-subsumes a clause D, denoted C ^ D. if and only if thcro exists a 
substitution such that C6 C D. Two clauses C and D are equivalent under 6-subsumption, 
denoted C ~ D. if and only if C ^ D and D ^ C. 

^-subsumption is reflexive and transitive. Two clauses may be equivalent under 6- 
subsumption without being variants. Two clauses C and D are variants, denoted C c::^ D, 
if they are equal up to variable renaming. 

Example Consider the following clauses: 

C = {p{x) ^ q{x,y), q{y,z), q{z,w), q{w,x) ), 
D = ( p{x) ^ q{x, y), q{y, x), q{x, x) ), and 
E = ( p(x) <— q(x. x) ). 

We have C ^ D since C{z/x,w/y \ CD.D<E since D{ylx} C E, and thus C <E. We 
also have E < D. Hence, D ^ E and still D E. 

Theorem 1 states that 6-subsumption between clauses is decidable. This was first shown 
by Robinson (1965, page 39). 

Theorem 1 (Decidability of ^^-subsumption between clauses) Let C and D he 

clauses. Then there exists a procedure to decide if C < D. 

As mentioned in the introduction, we are particularly interested in least general gener- 
alizations. The main reason is that a least general generalization includes the information 
of all consistent generalizations. 

Definition A clause C is a generalization under 6 -suh sumption of a set of clauses S = 
{Di,...,Dn} if and only if, for every 1 < i < n, C < Di. A generalization imder 6- 
subsumption C of S is a least general generalization under 6-subsumption (LGGO) of S if 
and only if, for every generalization under 6-subsumption C of 5, C :< C . 

Example Consider the following clauses: 

C = {p{a) ^ q{a), q{b) ), 

D={p(b)^q{b), q(x)), 

E = {p{y) ^ qiy), qib) ), and 

F = {p{y) ^ qiy), qib), qiz), q{w) ). 

Both clauses E and F are LGG6s of {C, D}. 
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In general, an LGG0 is not unique, as shown by the example above. However, it is 
unique up to ^-subsumption equivalence. Plotkin has shown (1971a, page 82) that there 
exists an LGG^ of every finite set of clauses. 

Theorem 2 (Existence of LGG^s) Let S be a finite set of clauses. Then there exists an 
LGG6 ofS. 

An LGG^ of a set of clauses is computable, and Plotkin has described (1971a) an algo- 
rithm for that. This algorithm is quite simple and easy to implement, but computationally 
expensive. 

2.2 Generalization under Implication 

Implication is the most natural and straightforward basis for generalization in inductive 
learning, since the concept of induction can be defined as the inverse of logically entailment. 

Definition A clause C implies a clause Z), denoted C =^ Z), if and only if every model for 
C is a model for D ({C} \= D). Two clauses C and D are equivalent under implication., 
denoted C ^ D, if and only if C =^ D and D ^ C . 

It is well-known that implication is reflexive and transitive. Two clauses may be equiv- 
alent imder implication without being equivalent under ^^-subsumption. 

Example Consider the following clauses: 

C = ( p{x, y, z) ^ p(y. z. x) ), and 
D = {p{x,y,z) ^p{z,x,y) ). 

Then we have C D, since D is a resolvent of C resolved with itself, and C is a resolvent 
of D resolved with itself. We also have C qk D, and even C ^ D. 

It has been claimed that implication and 6-subsumption are eciuivalent for function-free 
clauses (Helft, 1987). This is wrong as shown by the example above. The above example 
also shows that if a clause C implies a clause D then C does not necessarily ^-subsume 
D. It is well-known that implication is a strictly weaker relation between clauses than 
0-subsumption. 

Proposition 3 Let C and D be two clauses. If C < D then C =^ D. 

Proposition 3 has been proved by Idestam-Almquist (1993a, page 21). Unfortunately 
implication between clauses is problematic since it is undecidable, which has been proved 
by Schmidt-Schauss (1988, page 294). 

Theorem 4 (Undecidability of implication between clauses) LetC and D be clauses. 
Then there exists no procedure to decide if C =^ D. 

Niblett (1988) has claimed that implication between Horn clauses is decidable. This 
result has later been proved to be false (Marcinkowski & Pacholski. 1992). 

The definition of a least general generalization under implication (LGGI) follows the 
definition of an LGG6. 
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Definition A claiisr C is a generalization under implication of a set of clauses 5 = 
{Di Dn} if and only if, for every 1 < ' < »• C =^ Di. A generalization under im- 
plication C of 5 is a least general generalization under implication (LGGI) of S if and only 
if, for every generalization under implication C" of 5, C" =^ C. 

Example Consider the following clauses: 

C = (p{f(a))^p(a)), 
D = {p{P{h))^p{b)l 
E = ( p{f(x)) ^ p(y) ), and 
F = (p{f(z))^p(z)). 

The clause E is an LGG^ of {C,D}, and F is an LGGI of {C,D}. The LGG^ (clause 
E) is strictly more general than the LGGI (clause F). both under implication and under 
0-subsumption, since E =^ F but F E, and E < F but F E. 

Whether there exists an LGGI of every finite set of clauses is still an open problem. 
However, since implication between clauses is undecidable, it is clear that in general an 
LGGI is not computable. 

2.3 T-implication 

Because implication between clauses is imdecidable. w-e here introduce a stronger form 
of implication called T-implication, which is decidable between clauses. It is called T- 
implication since it is defined w.r.t. a finite set of ground terms T. In our presentation 
we use the notions of instance set of clauses, Skolem substitution, and term set of sets of 
clauses. 

Definition Let C be a clause, {xi, . . . , x.^} the set of variables in C, and T a set of terms. 
Then the instance set X(C, T) of C w.r.t. T is {C6 \ 6 = {xi/ti, . . . , Xn/tn} where {ti, . . . , 

tn} C T}. 

Definition Let cr be a substitution, C a clause, {xi, . . . , x^} the set of variables occurring 
in C S a set of clauses, and F the set of function symbols occiuring in 5 U {C}. Then a is 
a Skolem substitution for C w.r.t. S if and only if {xi/ai, . . . ^Xn/cLn} C a where 
are distinct constants, and F D {ai, . . . , = 0. 

Definition Let {Di, . . . , Z)„} be a set of clauses such that Di, . . . , Dn have no variables in 
common, 5 be a set of clauses, a a substitution, and T a set of terms. Then T is a term 
set of {-Di, . . . , Dn} by a w.r.t. S if and only if: 

a) cr is a Skolem substitution for {Di, . . . , Dn} w.r.t. 5, and 

b) T is finite and includes all terms and subterms occurring in {Di, . . . , Dn}cr. 

If T is equal to the set of terms and subterms occurring in {Di, . . . ,Dn}cr then T is a 
minimal term set of {Di, . . . ,Dn} by a w.r.t. S. 

Definition Let C and D be clauses, and T a term set of {D} by a w.r.t. {C}. Then C 
T-implies D w-.r.t. T. denoted C =^t D. if and only if T(C, T) \= Da. Tw-o clauses C and 
D are equivalent under T-implication w.r.t. T', denoted C D, if and only if C =^t' D 
and D =^t' C, where T' is a term set of {C,D}. 
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Note that the definition of T-imphcation is independent of the choice of the Skolem 
substitution rr. In the following, if we say that a clause C T-implios a clause D without 
explicitly stating T, we mean that C T-implies D w.r.t. a minimal term set of {-D}. Note 
that if C T-implies D w.r.t. a minimal term set of D then C T-implies D w.r.t. any term 
set of D. 

Example Consider the following clauses C and Z), substitution cr, set of terms T and set 
of clauses X(C, T): 

C = { p{f{x)) ^p{x) l 
D = ip(f(y))^p(y)), 

= {y/a}, 

T = {a,f(a),p(a)}, and 
J(C,T) = {(p(/(a))-p(a)), 

(p(f{a))^p(f(a))), 

(p(f{a))^p(f(a)))}. 

Then T is a minimal term set of {D} by rr w.r.t. {C}, and X(C, T) is the instance set of 
C w.r.t. T. We have that I(C,T) \= Da and thus C D. Note that C ^ D, and that 
C :^D. 

Like implication, T-implication is reflexive, but imlike implication, T-implication is 
not transitive (Idestam-Almquist, 1993a). The relationship between implication and T- 
implication, described in Corollary 6 below, follows from Herbrand's theorem. For a proof 
of Herbrand's theorem the reader is referred to a book by Chang and Lee (1973, page 61). 
In our proof of Corollary 6 we use the notion of the complement of a clause. 

Definition Let C = {Ai,...,Afn <— Bi, . . . , B^) be a clause, T a set of clauses, and 
a = {xi/ai, . . . , ajfc/a^} a Skolem substitution for C w.r.t. T. Then the set of ground unit 
clauses {(■;— Ai)a, . . . , (■;— Am)cr, (Bi <— )cr, . . . , (Bn <— )c} is the complement C of C by cr 
w.r.t. T. 

Theorem 5 (Herbrand's theorem) A set of clauses S is unsatisfiable if and only if there 
exists a finite unsatisfiable set S' of ground instances of clauses in S. 

Corollary 6 (Relationship between implication and T-implication) Let C and D 

be clauses. Then: 

a) if C =^T D for some term set T of {D} then C =^ D, and 

b) if C =^ D then there exists a term set T of {D} such that C =^t D. 

Proof: a) If C D then I(C, T) ^ Dcr, where T is a term set of {D] by a w.r.t. 
{C}. Hence, X(C, T) U Z) ^ _L, where D is the complement of D by cr. By Theorem 5, 
{C} U "D ^ ±, and thus C ^ D. 

b) If C =^ Z) then {C} U Z) |= ±, where D is the complement of D by a w.r.t. {C}. Then 
by Theorem 5, there exists a term set T of {Z)} such that T(C,T) U Z) ^ ±, and thus 
I(C, T) ^ Da. Then by definition C D. □ 
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It follows from Corollary 6 that T-implication can become an arbitrary good approx- 
imation of implication by extending the considered term set. T-implication is a strictly 
stronger relation between clauses than implication. The following example illustrates that 
if a clause C implies a clause D then C does not necessarily T-imply D. 

Example Consider the following clauses C, D and and set of terms T: 

C = {p{f{:c).y) ^p(z,x) ). 

D = (p(f(x).y) ^p(z. w) ). 

E = {p{f{a),a) ^p{a,f{a)) ), and 

T = {aJ(a)}. 

Then C =^ E since Z) is a resolvent of C resolved with itself and E is an instance of D. 
The set of terms T is a minimal term set of E. We do not show here the whole set T), 
but just point out that X(C, T) y= E and thus C E. However if we extend T to 
T = {a, /(a), /2(a)} then I{C,T') \= E, and thus C ^t' E. 

Below we show that if a clause C ^-subsumes a clause D then C also T-implies D. Thus, 
T-implication is a strictly weaker relation between clauses than 0-subsumption. We also 
show decidability of T-implication between clauses. 

Proposition 7 Let C and D be clauses and T a term set of {D}. If C ^ D then C =^t E>. 

Proof: If C ^ D then there exists a substitution such that C6 C D. Let T be a term 
set of {D} by a w.r.t. {C}. Then we have C9a E 'I(C,T). We also have C6a C Da, 
and thus C6a :< Da. Then by Proposition 3, C6a =^ Da ({C6a} \= Da). Consequently 
I(C, T) ^ Da, and then by definition C D. □ 

Theorem 8 (Decidability of T-implication between clauses) Let C and D be clauses 
and T a term set of {D}. Then there exists a procedure to decide if C =^t E>. 

Proof: By the definition of T-implication we have T) |= Da where T is a term set of 
D hy a w.r.t. {C}. We have that I{C,T) is a set of ground clauses and Da is a ground 
clause. Thus, it follows from the decidability of logical consequence in propositional logic 
that T-implication is decidable. □ 

2.4 Generalization under T-implication 

A least general generalization under T-implication (LGGT) is defined similar to an LGG0 
and an LGGI. 

Definition Let C be a clause, S = {Di. . . . , _D„| a set of clauses, and T a term set of S 
w.r.t. {C}. Then C is a generalization under T-implication of S w.r.t. T if and only if, for 
every 1 < i < n, C =^t Di. A generalization under T-implication C of 5 w.r.t. T is a least 
general generalization under T-implication (LGGT) of S w.r.t. T if and only if, for every 
generalization under T-implication C of S w.r.t. T, C =^t' C, where T' is a minimal term 
set of C. 
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Example Consider the following clauses: 



C 
D 
E 
F 



{p{f{a)) ^p{a) ), 
{p{P{h))^p{b)l 



{p(f(x)) *-p{y) ), and 



The clause E is an LGG6I of {C. D\. and F is both an LGGI and an LGGT of {C, D}. The 
LGGT is strictly more specific than the LGG0, since E =^ F and F ^ E. 

Below we prove that there exists an LGGT of every finite set of clauses. In fact we 
prove something stronger, namely that there exists, what we call, a complete LGGT of 
every finite set of non-tautological clauses. Note that a complete LGGT is 6-subsumed by 
any other generalization under T-implication. 

Definition Let C be an LGGT of a set of clauses S w.r.t. a term set T. Then C is a 
complete LGGT of S w.r.t. T if and only if, for every generalization under T-implication C 
of S w.r.t. T, C ^ C. 

If C is a clause then we let C"*" denote the set of positive literals in C, and C~ the set 
of negative literals in C. The following proposition has been proved by Gottlob (1987, page 



Proposition 9 Let C = C+ UC" be a elause and D = UZ) a non-tautologieal elause. 
IfC^D then C+ ^ D+ and C ^ D' . 

Lemma 10 Let S he a finite set of non-tautologieal elauses, T = {ti, . . . ,tm} a term set 
of S, V = {xi, . . . , Xm} a set of variables, and G = {Ci. C2. . . .} the (possibly infinite) set 
of all generalizations under T-implieation of S w.r.t. T. Then the set G' = T(Ci,V) U 
T{C2, V) U . . . is a finite set of clauses. 

Proof: Let d be the maximal depth of a clause in S, and Fs and Fq the sets of predicate 
and function symbols occurring in the clauses in S and G respectively. Then Fq U y is 
the set of variables, predicate and function symbols occurring in the clauses in G' . By 
Corollary G. G is a set of generalizations imder implication of S. Then, by Proposition 9 
and the definition of ^-subsimiption. Fq C Fs and the maximal depth of a clause in G is d. 
Hence Fq U F is finite and the maximal depth of a clause in G' is d, and consequently G' 
is a finite set of clauses. □ 

Lemma 11 Let C be a clause, S a set of clauses, V = {xi, . . . ,Xm} a. set of variables, 
and T = {ti, . . . ,tm} a term set of S by a w.r.t {C\, such that C is a generalization under 
T-implication of S w.r.t. T. Then there exists an LGG6 E of T(C,V) such that E is a 
generalization under T-implication of S w.r.t. T. 

Proof: Let T(C,V) = {Cpi, . . . ,Cpk}. Then, pi,...,pk are variable-pure substitutions, 
and for every LGG^ F of X(C,T) and every 1 < i < k. we have C ^ F and F ^ Cpi. Then, 
there exists an LGG^ E of X(C,T) and variable-pure substitutions 61,. . . ,6k such that, for 
every 1 < i < k, EOi C Cpi. 



110). 
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Let 7 = {xi/ti. Pm/^m}- an(l then T(C.T) = {C pi^, . . . ,Cpk'^}. Sinro C is a gener- 
alization imdor T-implication of S w.r.t. T, we have {C pi-j , . . . , C p^-j} \= Sa. For every 
1 < i < A;, EOi-j ^ Cpij. Then, for every 1 < « < A;, by Proposition 3, EOi-y =^ Cpi-j, 
and thus {_E6i7, . . . , EOk-y} \= {Cpi-y, . . . , Cj0fc7}. Since 6i, ... ,9k are variable-pure substi- 
tutions, we have {EOij, . . . , E6kj} C 1{E,T). Thus, T{E,T) \= {Cpi7, . . . , Cpfc7}, and 
T) 1= 5(7. Consequently i? is a generalization under T-implication of S w.r.t. T. □ 

Lemma 12 Ltt C, D and E be clauses such that C and D have no variables in common, 
and let T be a term set of {E} by a w.r.t. {C,D}. If C =^t E and D =^t E then 
CUD^T E. 

Proof: If C ^ and D E then by definition I(C, T) ^ Ea and 1{D, T) ^ Ea. Let 
X(C', T) = {Ci, . . . , Cn} and 1{D, T) = {A, . . . , D^}. Then I{C UD,T) = {C, U i:'^ | 1 < 
i < n and 1 < j < m}. Let / be a model for X(C U D,T). Then, for every 1 < i < n and 
1 < j < m, I is a model for Ci U -Dj. Hence if I is not a model for Ci for some 1 < « < n 
then / must be a model for Dj for every 1 < J < m. Then it follows that either / is a 
model for T) or / is a model for T), and thus / is a model for Ea. Consequently, 

1{C U D, T) ^ Ea, and then by definition CUD^t E. D 

Theorem 13 (Existence of complete LGGTs) Let S be a finite set of non-tautological 
clauses, and T a term set of S. Then there exists a complete LGGT of S w.r.t. T. 

Proof: Let T = {ti, . . . ,tm\, V = {xi, . . . , x^} be a set of variables, and G = {C'l, C'2, . . .} 
the (possibly infinite) set of all generalizations under T-implication of S w.r.t. T. By Lemma 
10, the set Gr' = I{Ci, V) UT(G'2- V) U ... is a finite set of clauses. Since G' is finite, the set 
{I(Ci,V),I{C2.V). . . .} is also finite. 

For every « > 1, by Lemma 11, there exists an LGGd Ei of I^GijV) such that Ei is a 
generalization under T-implication of S w.r.t. T. Then rename the variables in Ei,E2, . . ■ 
such that, for every k > 1 and p > 1, Ek = Ep whenever I(Ck,V) = I(Cp,V) and 
otherwise Ek and Ep have no variables in common. Then the set {Ei, E2, ■ ■ ■} is finite, 
since V),'I(G2, V), . . .} is finite. Let F = Ei U E2 U . . which consequently is a 

clause. 

For every i > 1, by the definition of an LGG6, Gi :< Ei, and thus Gi :< F. Then, for 
every « > 1, by Proposition 7, Gi =^t E. As showed above, for every i > 1, Ei is a 
generalization under T-implication of S w.r.t. T. Then, by Lemma 12, _F is a generalization 
under T-implication of S w.r.t. T. Consequently, _F is a complete LGGT of S w.r.t. T. □ 

Theorem 14 (Existence of LGGTs) Let S be a finite set of clauses, and T a term set 
of S. Then there exists an LGGT of S w.r.t. T. 

Proof: Let D be a tautology and a a Skolem substitution for D. Then T \= Da, and 
thus for every clause G, G E>. If every clause in 5 is a tautology, then every clause 
is a generalization under T-implication of S w.r.t. T, and every tautology is an LGGT of 
S w.r.t. T. Let S' be the set of clauses obtained from S by removing all tautologies. It 
is clear that every generalization under T-implication of S w.r.t. T also is a generalization 
under T-implication of S' w.r.t. T. By Theorem 13, there exists an LGGT of S' w.r.t. T. 
Consequently there exists an LGGT of 5. □ 
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3. Reduction of Implication to ^-subsumption 

There are generalizations under implication that are not generalizations under 0-subsump- 
tion. Oiu" main idea to find all generalizations imder implication, is to reduce implication 
to ^-subsumption, which can be achieved by inverting self-resolution. In this section we will 
describe a technique for inverting resolution based on or-introduction of literals. We will 
also introduce the notion of expansion of clauses, which summarizes our idea of reduction 
of implication to 0-subsumption. 

3.1 Difference between ^^-subsumption and Implication 

In section 2.2, we showed that C =^ D follows from C ^ D, but not the converse. 
Hence, there are generalizations under implication that are not generalizations under 6- 
subsumption. It follows from a result by Gottlob (1987) that the difference between 6- 
subsumption and implication only concerns ambivalent clauses, as defined below. 

Definition A clause C is ambivalent if and only if there exist a positive literal A £ C and 
a negative literal B £ C such that A and B have the same predicate symbol. 

Example The clause 

C=(p(f(a))^q(b),p(a)) 

is ambivalent since p(f'^(a)) and ^p(n) have the same predicate symbol. However, C is not 
recursive since neither p[a) nor q(b) is unifiable with a variant of p( f^(a)). 

Proposition 15 Lei C be a clause and D a non-ambivaleni clause. Then C =^ D if and 
only ifC ^ D. 

Proposition 15 has been proved by Gottlob (1987, page 110). It follows from this 
proposition that an LGG0 and an LGGI of a set of clauses, including at least one non- 
ambivalent clause, are ecjuivalent. 

Muggleton (1992) has investigated the relationship between resolution and implication 
between clauses. He describes the subsumption theorem (Lee, 1967) in terms of input 
resolution, and gives a corollary about the relationship between 6-subsumption and impli- 
cation betw-een clauses. Unfortimately. this formulation of the subsumption theorem, which 
later also has been used by Idestam-Almquist (199.3c, 1993a), has been shown to be wrong. 
Nienhuys- Cheng and de Wolf (1995) have given a counter-example which shows that the 
subsumption theorem for input resolution does not even hold in the special case where the 
considered set of clauses contains only one clause. Below we give the correct formulation of 
the subsumption theorem, which is based on the nth resolution (Robinson, 1965). 

Definition A substitution is a unifier for a finite set of literals S if and only if S6 is a 
singleton. A unifier 6 for 5 is a most general unifier (mgu) for S if and only if for each 
unifier a oi S there exists a substitution 7 such that a = 6^. 

Definition Let C be a clause, FCC and 7 an mgu of T. Then C7 is a factor of C. 
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Definition A clause i? is a resolvent of two clauses C and D if and only if there are C-j, 
Di^i. A. B and 6 such that: 

a) C7 is a factor of C and D/j is a factor of D, 

b) C7 and D/i have no variables in common, 

c) ^ is a literal in C7 and i? is a literal in Dfi, 

d) 6 is an mgu of {^4. B}, and 

e) R is the clause ((C7 - {A}) U (D^ - {B}))e. 
The clauses C and D are called parent clauses of i?. 

Definition Let T be a set of clauses. Then, the nth resolution of T, denoted 7^"(T), is 
defined as: 

a) 7^°(T) = T, and 

b) 7^'^(T) = 7^''-l(T) U {i? I C, D e 7^"-^(T) and i? is a resolvent of C and D} if n > 0. 

Theorem 16 (Subsumption theorem) LetT be a set of clauses and C a non-tautological 
clause. Then T \= C if and only if there exists a clause D £ 7^"(T) such that D <C for 
some n > 0, 

Two different recent proofs of Theorem 16 have been presented, one by Nienhuys-Cheng 
and de Wolf (1995), and one by Bain and Muggleton (1992). There also exist at least two 
different earlier proofs of this theorem in the literature, one by Slagle. Chang and Lee (1969), 
and one by Kowalski (1970). We are interested in the number of resolutions involved in the 
computation of a clause, and therefore we introduce the notion of nth resolution layer. A 
clause in the nth resolution layer has been obtained from the original set of clauses by n — 1 
resolutions. 

Definition Let T be a set of clauses. Then, the nth resolution layer of T, denoted £"(T), 
is defined as: 

a) £i(T) = T, and 

b) C^{T) = {i? I i? is a resolvent of C G £"(T) and D G £*'(T) where m+p = n-l, m>l 
and p > 1} if n > 1. 

Corollary 17 (Implication between clauses using resolution) Let C he a clause and 
D a non-tautological clause. Then C =^ D if and only if there exists a clause E G £"({C}) 
such that E < D for some n > 1. 

Corollary 17 follow-s from Theorem 16, and the observation that, for every n > 1, if a 
clause C G C'^{T) then also C G TZ'^{T). This corollary tells us that implication between 
clauses is equivalent to a combination of self-resolution and 0-subsumption. Muggleton 
(1992) has introduced the notion of powers and roots of clauses for specializations and gen- 
eralizations of clauses where the clauses are resolved with themselves. Below we present 
definitions of these and related concepts modified w.r.t. the correct definition of the sub- 
sumption theorem. 

Definition A clause D is an nth power of a clause C if and only if Z) is a variant of a clause 
in £"({C}) (n > 1). We also say that C is an nth root of D. A clause D is an indirect nth 
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power of a claiisr C if and only if there exists a clause E snch that E < D and E is an nth 
power of C . We also say that C is an indirect nth root of D. 

Let C be a clause and D an indirect /ztli power of C . Then D is a proper indirect nth power 
of C if and only \i C D. We also say that C is a proper indirect nth root of D. 

Example Consider the following clauses: 

C = {p{f{x)) ^p(x)), 

D = (p{f{x))^p{x)). 

E = { p{ f{x)) ^p{x)), 

F = { p{P{a)) ^p{a), p{b) ), and 

G = (pix)^p(b)). 

The clause C is a second root of D, and a third root of E. The clause C is also an indirect 
second root of F, since C is a second root of D and D ^-subsimies F. In fact C is a proper 
indirect second root of F. since C ^ F. For every n > 1, the clause G is an indirect nth 
root of itself, but none of these indirect roots is a proper indirect root. 

To say that a clause implies another non-tautological clause or to say that the clause 
is an indirect root of the other clause, is equivalent. However, to say that a clause is an 
indirect nth root for some specified n is more informative. 

Implication between clauses can be described as a combination of self-resolution and 
^-subsumption. Plotkin's algorithm to compute LGG^s gives us a suitable tool for find- 
ing generalizations imder ^-subsumption. Hence, to be able to find generalizations under 
implication we also need a technique to invert resolution. 

3.2 Inverting One Resolution by Or-introduction 

Other work on inverting resolution has primarily considered the problem of constructing 
one parent clause given the resolvent and the other parent clause (Muggieton & Bimtine, 
1988; Rouveirol k. Puget, 1989; Wirth, 1989; Muggieton, 1990; Hume k Sammut, 1991; 
Idestam-Almquist, 1992; Rouveirol, 1992). Below we will describe how or-introduction can 
be used to construct two parent clauses from only the resolvent. Let C and D be clauses, 
and the following clause R a resolvent of C and D: 

R = ((C7 - {A}) U {D^l - {B}))e, 

where C7 is a factor of C, Dfi is a factor of D, A £ C7, B £ Dp. and 6 is an mgu for 
{A,B}. We seek parent clauses of R that are minimally general. Then we should let 7, p 
and be empty substitutions, which corresponds to an assumption that no instantiation of 
variables has been done in the resolution of C and D, and thus we have A = B. We should 
also let C — {A} = D — {B}, which corresponds to an assumption that each literal in R is 
inherited both from C and D. Then we have 

C = RU{A} and D = RU {A\. 

where A could be any literal, and we say that C and D are obtained from R by or- 
introduction of the literal A. 
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Example Consider the following clauses: 

C = (p{f(a))^p(f(a)),p(a)), 

D = (p(f(a)).p(f{a)) ^p(a)), and 

R=ip{P{fiy)^P(a)). 

The clauses C and D are parent clauses of R. 

The above technique to find parent clauses can be used to reduce implication to 6- 
subsumption. This is of interest for ambivalent clauses, such as R in the example above, 
for which there are proper indirect roots. For example the clause 

G = {p{f{x)) ^p{x) ) 

is a proper indirect second root of R, and it ^-subsumes both C and D. 

Proposition 18 shows that the set of two clauses obtained by our technique for inverting 
one resolution is logically equivalent to the original clause. In Proposition 19 it is shown 
that or-introduction of one literal is a general technique for inverting one resolution. 

Proposition 18 Let R be a clause and L a literal. Then {R} = {RU {L},i?U {i^}}. 

Proof: Since R C RU {L} and R C RU {L}, we have R ^ RU {L} and R ^ RU {L}. 
Then by Proposition 3, R ^ RU {L} and R ^ RU {L}. Thus {R} \= {RU {L}, R U {L}}. 

The clause i? is a resolvent of i? U {L} and i? U {L}. Then by soimdness of resolution, 
{R U {L}, R U {I}} \= {R}. Consequently, {R} = {RU {L}, R U {I}}. □ 

Proposition 19 Let C and D he clauses and R a resolvent of C and D. Then there exists 
a literal L such that C ^ R U {L} and D ■< RU {L}. 

Proof: By the definition of a resolvent we have R = ((C7 — {A}) U (Dfi — {B}))9, where 
C7 is a factor of C, Dfi is a factor of D, A £ C7. B G Dfi and AO = DO where 6 
is an mgu for {A.'B}. Let L = A6, and then R U {1} = [C-f U {Dpi - {D}))6, and 
RU(L] = ((C7 - {,4}) U Dn)e since AO = BO. Hence C-^0 C (C7 U {Dp - {B]))e and 
DjJtO C ((C7 - {A}) U Dii)6, and consequently C < RU {L} and D < RU {!}. □ 

3.3 Inverting Multiple Resolutions by Or-introduction 

The technique for inverting one resolution can be generalized to a technique for inverting a 
sequence of resolutions as follows. The set 

{i?U{Li},i?U{I7}|. 

where i? is a clause and Li is a literal, is a set of two clauses from which the clause R follows 
by one resolution. Similarly, the two sets 

{R U{Li]U {L2}^ U {Li} U {Ill^i? U {17}}, and 
{R U {Li}, i? U {Li} U {L2}, i? U {Li} U {L2}}, 

where i? is a clause and Li and L2 are literals, are sets of three clauses from which R follows 
by two resolutions. In the same way. for given literals Li, L2 and L3, we have six different 
sets of four clauses from which R follows by three resolutions, and so on. All these sets are 
or-introduced from the clause R. 
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Definition Let C he a clause and Q a sequence of literals. Then a set of clauses S is 
or-inirvduced from C hy if and only if either: 

a) S = {C} and = [], or 

b) S = (S' -{D})U{DU{L},DU{L}} and = [Li, . . . , L], where S' is a set of clauses 
or-introduced from C by [Li, . . . , L„] and D £ S'. 

Example Consider the following clauses: 

C=(p(f{a))^p(a)), 

Di = (p(f(a)),p(f(a))^p(a)). 

D2 = (p(f(a))^p{f(a)),p{a)). 

El = ( p( fia)), p( f(a)), p{f(a) ^ p{a) ), and 

E2 = (p(fia)), p{P{a)) ^p(f(a)), p{a) ). 

The set of clauses {Di,D2\ is or-introduced from C by \p{P {a))], and the set of clauses 
{-El, E2} is or-introduced from Di by Consequently, the set of clauses Ei, E2} 

is or-introduced from C by [p(f^(a)), p(f(a))]. 

In the example above, clause Di is a resolvent of Ei and E2, and C is a resolvent of Di 
and 1)2 ■ Consequently, C is derivable from {D2, -^1,-^2} by resolution. That a set of clauses 
or-introduced from a clause is logically equivalent to the clause, is shown by the following 
theorem. 

Theorem 20 (Equivalence preservation of or-introduction) Let S be a set of clauses 
or-introduced from a clause C by a sequence of literals [Li, . . . , Then S = {C}. 

Proof: The proof is by mathematical induction on n. It should be noted that S, in the 
statement of the theorem, in the proof is indexed by n. 

Base step (n—0): So is or-introduced from C by []. Hence So = {C}. 

Induction hypothesis (n—k): Sk = {C}, where Sk is or-introduced from C by [Li, . . . ,Lk]. 

Induction step (n-k+1): Let D £ Sk- Then Sk+i = {Sk - {D}) U {DU {Lk+i} , DU {Lk+i}} 
is or-introduced from C by [Li, . . . , L^, L^+i]. By Proposition 18. we have {Z)U {L^+i}. DU 
{Lfc+i}} = {-D}, and consequently Sk+i = Sk- By the induction hypothesis Sk = {C}, and 
thus Sk+i = {C}. □ 

In section 3.2 we showed that it is possible to invert one resolution by or-introduction 
of one literal. Below we show that it is possible to invert a sequence of resolutions by 
or-introduction of a sequence of literals. 

Lemma 21 Let D and E be clauses, {Ci.....C„} a set of clauses, and {Di. . . . . Dn} a 

set of clauses or-introduced from D, such that D E and, for every 1 < / < n, C'i ^ Di. 
Then there exists a set of clauses {Ei,...,En} or-introduced from E, such that for every 
I <i <n, C, < E,. 

Proof: Let Di be an arbitrary clause in {Di, . . . , Dn}- Then we have Di = DUAi for some 
set of literals Ai, since {Di, . . . , D^} is or-introduced from D. Since Ci ^ Di, there exists 
a substitution 6i such that CiOi C DU Ai. Since D < E, there exists a substitution a such 
that Da C E- Thus we have {D U Ai)a C (_E U AjCr), and consequently CiOiO C E U AjCr. 
Let Ei = El] AjCr and we have Ci -< Ei- □ 
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Theorem 22 (Inverting resolution using or-introduction) Let T be a set of clauses, 
D a clause in C^{T). Then there exists a set of clauses S or-introduced from D such that 
for each E £ S there exists a clause C £ T such that C ^ E. 

Proof: The proof is by complete mathematical induction on n. It should be noted that D 
and S, in the statement of the theorem, in the proof are indexed by n. 

Base step (n—1): By the definition of nth resolution layer C^{T) = T, and thus Di G T. 
We have that Si = {Di} is or-introduced from Di by the empty sequence of literals 0^ = []. 
Hence, for Di G there exists a clause Di G T such that Di :< Di. 

Induction hypothesis (n—k): For every 1 < i < k, there exists a set of clauses Si or- 
introduced from Di by some sequence of literals Oj- = [Li, . . . , such that for each 
E G Si there exists a clause C G T such that C < E. 

Induction step (n—k+I): By the definition of nth resolution layer, -Dfc+i is a resolvent 
of some clauses G C^iT) and Dp G C^(T) such that m+p = k, l<m<k and 
1 !^ i' !^ k- Then by Froposition 19, there exists a literal L such that D^ ^ D^+i U {L} 
and Dp ^ Dk+i U {L}. 

By the induction hypothesis, there exists a set of clauses Sm or-introduced from D^ by 
some sequence of literals 0^ = [^i, • • ■ ,^m-i] such that for each E G Sm there exists a 
clause C £ T such that C ^ E. Then by Lemma 21, there exists a set of clauses S'^ 
or-introduced from -Dfc+i U {L} by some sequence of literals = [A'l, . . . , ^^-i] such that 
for each E' G S'^ there exists a clause C GT such that C :< E'. 

By the induction hypothesis, there also exists a set of clauses Sp or-introduced from Dp by 
some sequence of literals Qp = [Bi, . . . , -Bp-i] such that for each E £ Sp there exists a clause 
C G T such that C ^ E. Then by Lemma 21, there exists a set of clauses S'p or-introduced 
from -Dfc+i U {L} by some sequence of literals 0^ = [B[, . . . , Bp] such that for each E' G -5^ 
there exists a clause C £ T such that C < E' . 

Then it follows from the definition of or-introduction that Sk-\.\ = S'^ U 5^ is a set of clauses 
or-introduced from Dfc+i by Ofc+i = [L. A'^, . . . , A'^, B[, . . . ^ Bp]. Consequently, there exists 
a set of clauses Sk+i or-introduced from -Dfc+i such that for each E G Sk+i there exists a 
clause C G T such that C ^ E. □ 

3.4 Expansion of Clauses 

In the section 3.3 it was described how a reduction of generalization can be achieved by 
replacing a clause by a set of clauses. Here we show how this set of clauses equivalently 
can be described by a single clause, which we call an expansion of the original clause. By 
definition, if a clause C 6-subsumes every clause in a set of clauses 5, then C will also 
^-subsume an LGG6 of S. This leads us to our definition of expansion of clauses. The idea 
of expansion of clauses was first presented by Idestam-Almquist (1993c). 

Definition Let D he a clause and a sequence of literals. Then a clause E is an expansion 
of Z) by n if and only if E is an LGGO of a set of clauses or-introduced from D by 0. 
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Example Consider the following clauses: 

C = {p(f(x)) ^p(x) ), 

D=(p(f(a))^p(a)), 

D^ = {pif(a))^p{p{a)),p(a)), 

D2 = {p{f(a)), p{f(a)), p(f(a) ^ p(a) ), 

D3 = (p{f(a)), p{f(a)) ^p(f{a)), p(a) ), and 

E = (p{f(x)), p{ f(a)) ^p(a), p{x) ). 

The set of clauses {-Di, -D2, ^3} is or-introduced from the clause D by [p(/^(a)), j>(/(a))], and 
E is an LGG0 of {-Di, -D21 -D3}. Consequently, E is an expansion of D by [p(/^(ci)), p{f{a))\. 

Note that implication has been reduced to 0-subsumption in the example above. We 
have C ^ D and C D, but for the expansion E of D we have C ^ E. 

Expansion can be regarded as a transformation technique, since the expansion of a clause 
is logically equivalent to the clause itself. 

Theorem 23 (Equivalence preservation of expansion) Let D be a clause, and E an 
expansion of D. Then E D. 

Proof: By the definition of expansion, we know that there exists a set of clauses S or- 
introduced from D hy Q such that E is an LGG^ of S. By Theorem 20, we have {D} = S. 
By the definition of an LGG6, E < F for each F £ S. Then by Proposition 3, E ^ F for 
each F £ S. Thus {E} \= S, and consequently E ^ D. 

We have D C F for each F £ S. Then by the definition of an LGG0, we have D < E, and 
by Proposition 3 D =^ E. Consequently, E ^ D. □ 

Below we prove that for every generalization under implication of a clause there exists 
an expansion of the clause such that the generalization under implication is reduced to a 
generalization under 0-subsumption. 

Theorem 24 (Reduction of implication to ^-subsumption using expansion) Let 

C he a clause and D a non-tautological clause such that C ^ D. Then there exists an 
expansion E of D such that C ^ E. 

Proof: By Corollary 17. there exists a clause D' G £"({C'|) such that D' ^ D for some 
n y 1. By Theorem 22, there exists a set of clauses S' or-introduced from D' such that for 
each F' £ S' we have C ^ F' . Then it follows from Lemma 21 that there exists a set of 
clauses S or-introduced from D such that for each F £ S we have C ^ F. Then let E be 
an LGG^ of S, and thus an expansion of D, and we have C :< -E by the definition of an 
LGG6. □ 

3.5 Complete Expansion 

Generalizations under implication of a clause can be reduced to generalizations under 6- 
subsumption of an expansion of the clause. We are particularly interested in expansions 
of clauses such that every generalization under implication is reduced to a generalization 
under 0-subsumption of that particular expansion. 
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Definition Lot D be a clause, and E an expansion of D. Then i? is a complete expansion 
of D if and only if, for every clause C, C ^ E whenever C ^ D. 

Eccall that an expansion of a clause is a clause, and thus finite. Muggleton and Page 
(1994. page 166) has shown that complete expansions, which they call finite self-saturations, 
do not exist for all clauses. 

Theorem 25 (Non-existence of complete expansions) There exist non-tautological 
clauses for which there exist no complete expansions. 

The non-existence of complete expansions is due to that for some clauses there are 
infinitely many distinct generalizations under implication. Because of this we turn to the 
problem of reducing every generalization under T-implication to a generalization under 
0-subsumption of a single expansion. 

Definition Let _D be a clause. E an expansion of D and T a term set of {D}. Then E is 
a T-complete expansion of D w.r.t. T if and only if, for every clause C, C :< E whenever 
C^T D. 

Example Consider the following clauses: 

C2 = (piP(y))^p(y)), 

El = ( pifiy)). pi fHo)) ^ Pin). p{y) ). and 

E2 = (p(f(x)). pUHv))- pUHo)) ^p{a). p{y). p{x) ). 

The clauses Ci and C2 are proper indirect roots of D, such that Ci =^t D and C2 =^t 
D. The clause Ei is an expansion of D by \p(f^(a))], and Ei is an expansion of D by 
[p(/^(^))5 Pif'^io-))^ p(fio-))]- The expansion E2 is a T-complete expansion but Ei is not. 

In the example above the T-complete expansion E2 of D is also a complete expansion 
of D. However, in contrast to complete expansions, T-complete expansions exist for all 
non-tautological clauses. 

Theorem 26 (Existence of T-complete expansions) Let D be a non-tautological clause 
and T a term set of {D}. Then there exists a T-complete expansion E of D w.r.t. T. 

Proof: By Theorem 13, there exists a complete LGGT F of {D} w.r.t. T. Hence, for 
every clause C, if C =^t D then C < F. By the definition of a complete LGGT, we have 
F Z), and then by Corollary 6, F =^ D. By theorem 24, there exists an expansion E of 
D such that F ^ E. Thus, for every clause C, if C F> then C ^ E, and consequently 
i? is a T-complete expansion of D w.r.t. T. □ 

If w-e can compute T-complete expansions of a set of clauses then we can use Plotkin's 
algorithm for computing an LGG^ to compute an LGGT. From the proof of Theorem 26 
it follows that the candidate set of literals to be used to compute a T-complete expansion 
is finite. Since expansion is equivalence preserving we could simply test all different ways 
to expand a clause by sequences of literals from this candidate set. and in this way obtain 
a T-complete expansion. This is of course an extremely complex process, but at least 
theoretically, T-complete expansions and LGGTs are computable. 
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4. Concluding Remarks 

We have studied the problem of generahzation of clauses. In section 2, we described the 
framework for generalization of clauses developed by Plotkin (1970. 19711). 1971a), which is 
based on 6-subsumption. Implication is the most natural basis for in(hirtivr generalization. 
In section 2, we therefore also studied the theory of generalization under implication. 
The contents of section 2 can be summarized as follows: 

1. It is decidable whether a clause ^^-subsimics another clause. 

2. There exists a least general generalization under 0-subsumption (LGG0) of every finite 
set of clauses. 

3. It is undecidable whether a clause implies another clause. 

4. It is an open problem whether there exists a least general generalization under impli- 
cation (LGGI) of every finite set of clauses. 

5. T-implication is a strictly stronger relation between clauses than implication, and 
strictly weaker than 0-subsumption, and T-implication can become an arbitrarily 
good approximation of implication by extending the considered term set. 

6. It is decidable whether a clause T-implies another clause. 

7. There exists a least general generalization under T-implication (LGGT) of every finite 
set of clauses. 

In section 3, we studied the difference between ^-subsumption and implication on clauses. 
We presented our approach to find all generalizations imder implication, by reducing im- 
plication to 6-subsumption. This can be achieved by inverting self-resolution, and we de- 
scribed a technique for inverting resolution based on or-introduction of literals. We also 
described expansion of clauses, which summarizes our idea of reduction of implication to 
0-subsumption. 

The contents of section 3 can be summarized as follows: 

1. An expansion of a clause is an LGG0 of a set of clauses obtained by or-introduction 
from the clause. 

2. For every generalization under implication of a clause there exists an expansion of the 
clause, logically equivalent to the clause, such that the generalization under implica- 
tion is reduced to a generalization under 0-subsumption. 

3. There exist non-tautological clauses for which there exist no complete expansions, 
which means that there are no expansions of the clauses such that every general- 
ization under implication is reduced to a generalization under 6-subsumption of the 
expansions. 

4. For each non-tautological clause there exists a T-complete expansion, which means 
that every generalization under T-implication of the clause is reduced to a generaliza- 
tion under ^-subsumption of the expansion. 
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As notrd in section 3.5, T-complete expansions and LGGTs are computable, but such 
a computation is extremely costly. This is not sm'prising since oiu' framework for general- 
ization under implication is based on and extends Plotkin's framework for generalization 
under 0-subsumption, which already suffers from complexity problems. In general an LGG 
of a set of clauses may grow exponentially in the number of clauses in the set (Muggleton & 
Feng, 1990). Even an LGG reduced imder ^-subsumption, which means that all literals that 
are redundant under 6-subsumption are removed, may grow exponentially in the number of 
clauses (Kietz, 1993). Since an expansion of a clause is an LGG of a set of or-introduced 
clauses, the computational cost of an expansion grows exponentially in the number of literals 
used in the or-introduction. In the computation of a T-complete expansion a large mmiber 
of literals may be considered, and consequently such a computation would be extremely 
costly. 

However, although Plotkin's framework for generalization under 0-subsumption is com- 
putationally expensive, it has been widely used as a theoretical framework. Then to make 
it practical, a number of different restrictions on the clausal language has been considered, 
for example «j-determinacy (Muggleton & Feng, 1990). In a similar way we hope to find 
restrictions under which our here presented framework for generalization under implication 
can be practically useful. 

Idestam-Almquist (1993b, 1993a) has described a technique to efficiently compute a 
restricted form of generalizations imder implication. Recently. Muggleton has presented 
another approach based on generating a number of clauses, so called sub-saturants, which 
are candidates for being indirect roots, and then testing whether they are so or not (Muggle- 
ton, 1995). This approach might be a way to more efficiently compute some generalizations 
under implication. Some approaches to learn reciu'sive definitions (recursive logic programs) 
by generalization under implication have been presented (Lapointe & Matwin, 1992; Aha, 
Lapointe, Ling, & Matwin, 1994; Idestam-Almquist, 1995). These approaches are based 
on structural analysis of the given examples, but can theoretically be described in our 
framework. 

A study by Cohen (1995a, 1995b) of the learnability of recursive logic programs has 
previously been presented in this journal. In this study it was shown that a recursive logic 
program consisting of one constant-depth determinate closed fc-ary recursive clause and 
one constant-depth determinate non-recursive clause is PAC-learnable given an additional 
"base-case oracle", which determines if a positive example is covered by the non-recursive 
base clause of the target program alone. It was also shown that generalizing this class of 
learning problem in any natural way leads to a computationally difficult problem. This 
result tells us that to efficiently learn more complex recursive hypotheses some extra infor- 
mation, such as rule models (Kietz & Wrobel, 1992) or program recursion schemes (Hamfelt 
&L Nilsson, 1994), is needed. 

The contributions of this paper are threefold. First, we have systematically reviewed and 
discussed the concepts relevant to generalization in a first-order setting. Second, we have 
introduced T-implication, a stronger form of implication which is decidable between clauses. 
Third, we have further developed previous work of the author (Idestam-Almquist. 1993c) 
on extending Plotkin's framework for generalization under ^-subsumption to generalization 
under implication. 
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